Posted on | Edited on | In ,

The autumn in both Vancouver and Seattle is gorgeous…

Overview Seattle on Space Needle Around Space Needle
Overview Seattle on Space Needle Around Space Needle
Space Needle So Heavy
Space Needle So Heavy
Ferris Wheel Pier 55
Ferris Wheel Pier 55
Vancouver Maple UBC Poisonous Mushroom
Vancouver Maple UBC Poisonous Mushroom

Alright, let’s rapidly test Detectron2.

Installation is detailedly summarized in INSTALL.md.

We can simply follow GETTING_STARTED.md for some simple demonstrations. Make sure you’ve downloaded the demo pictures from Detectron1 demo and save under Detectron2’s folder demo.

1
2
3
4
5
6
7
8
longervision-GT72-6QE% python demo/demo.py --config-file configs/COCO-InstanceSegmentation/mask_rcnn_R_50_FPN_3x.yaml \
  --input ./demo/17790319373_bd19b24cfc_k.jpg \
  --output ./result.jpg \
  --opts MODEL.WEIGHTS detectron2://COCO-InstanceSegmentation/mask_rcnn_R_50_FPN_3x/137849600/model_final_f10217.pkl
[10/13 17:47:01 detectron2]: Arguments: Namespace(confidence_threshold=0.5, config_file='configs/COCO-InstanceSegmentation/mask_rcnn_R_50_FPN_3x.yaml', input=['./demo/17790319373_bd19b24cfc_k.jpg'], opts=['MODEL.WEIGHTS', 'detectron2://COCO-InstanceSegmentation/mask_rcnn_R_50_FPN_3x/137849600/model_final_f10217.pkl'], output='./result.jpg', video_input=None, webcam=False)
WARNING [10/13 17:47:01 d2.config.compat]: Config 'configs/COCO-InstanceSegmentation/mask_rcnn_R_50_FPN_3x.yaml' has no VERSION. Assuming it to be compatible with latest v2.
  0%|                                                                                                                                                                                        | 0/1 [00:00<?, ?it/s][10/13 17:47:05 detectron2]: ./demo/17790319373_bd19b24cfc_k.jpg: detected 13 instances in 0.29s
100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00,  2.59it/s]

Let’s take a look at the result:

17790319373_bd19b24cfc_k

And of course, my pictures taken in Seattle and Vancouver:

Overview Seattle on Space Needle Around Space Needle
Overview Seattle on Space Needle Around Space Needle
Space Needle So Heavy
Space Needle So Heavy
Ferris Wheel Pier 55
Ferris Wheel Pier 55
Vancouver Maple UBC Poisonous Mushroom

Oooooops… it seems there are still quite a lot of ERROR detections…
Anyway… tomorrow, 热干面. Google translator: Hot Noodles with Sesame Paste…

Posted on | In ,

Flying back to China soon. Before leaving, here comes a simple blog testing Tensorflow 2.0. In this blog, I strictly follow Amita Kapoor and Ajit Jaokar’s free book Getting Started with TensorFlow 2.0.

For simplicity, let’s try it out directly:

1. Tensorflow 2.0

2. Tensorflow Dataset

Posted on | In ,

Finally, I’m going to talk about Raspberry Pi a bit. It has been discussed so widely that NOTHING special should be further talked from me.

To write something about Raspberry Pi is to say GOOD BYE to my Raspberry Pi 3B, and WELCOME Raspberry Pi 4 at the same time. Our target today is to build an AI edge computing end as the following video:

Raspberry Pi 3B with Intel Movidius NCS and openVINO

1. Raspberry Pi 3B vs. Raspberry Pi 3B+

Before everythig starts, let’s carry out a simple comparison between Raspberry Pi 3B and Raspberry Pi 3B+:

Raspberry Pi 3B vs. Raspberry Pi 3B+

2. Movidius Neural Compute Stick on Raspberry Pi 3B

Then, we just follow the following 2 blogs Run NCS Applications on Raspberry Pi and Adding AI to the Raspberry Pi with the Movidius Neural Compute Stick to utilize these 2 outdated products:

Raspberry Pi 3B Intel Movidius Neural Compute Stick 1
Raspberry Pi 3B Intel Movidius Neural Compute Stick 1

Intel Movidius Neural Compute Stick 1 is NOT listed on Intel’s official website any more. But github support for Intel Movidius Neural Compute Stick 1 can be found at https://github.com/movidius/ncsdk.

2.1 Raspberry Pi 3B’s Wifi Module Does NOT Work Properly

2.1.1 Wifi NOT Working

I met a bug that I failed to find the solution forever. Three related forum issues are enumerated as follows, and you are welcome to carry out further investigation.

Anyway, ip -c address gives out the following ERROR message:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
➜  ~ ip -c address
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: enxb827ebc1b876: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether b8:27:eb:c1:b8:76 brd ff:ff:ff:ff:ff:ff
    inet 192.168.1.86/24 brd 192.168.1.255 scope global noprefixroute enxb827ebc1b876
       valid_lft forever preferred_lft forever
    inet6 2001:569:7efd:8200:6cce:f005:e59c:7a4a/64 scope global dynamic mngtmpaddr noprefixroute 
       valid_lft 7455sec preferred_lft 7155sec
    inet6 fe80::a3ea:1480:f310:3f65/64 scope link 
       valid_lft forever preferred_lft forever
3: wlan0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether b8:27:eb:94:ed:23 brd ff:ff:ff:ff:ff:ff

2.1.2 Use a Wifi USB Dongle

An arbitrary 802.11 N Wifi 2.4G USB dongle is able to activate Wifi for Raspberry Pi 3B in my case. BTW, mine is EDUP EP-N8508GS.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
➜  ~ ip -c address
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: enxb827ebc1b876: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether b8:27:eb:c1:b8:76 brd ff:ff:ff:ff:ff:ff
    inet 192.168.1.86/24 brd 192.168.1.255 scope global noprefixroute enxb827ebc1b876
       valid_lft forever preferred_lft forever
    inet6 2001:569:7efd:8200:6cce:f005:e59c:7a4a/64 scope global dynamic mngtmpaddr noprefixroute 
       valid_lft 7470sec preferred_lft 7170sec
    inet6 fe80::a3ea:1480:f310:3f65/64 scope link 
       valid_lft forever preferred_lft forever
3: wlan0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether b8:27:eb:94:ed:23 brd ff:ff:ff:ff:ff:ff
4: wlxe84e0635166d: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
    link/ether e8:4e:06:35:16:6d brd ff:ff:ff:ff:ff:ff
    inet 192.168.1.87/24 brd 192.168.1.255 scope global noprefixroute wlxe84e0635166d
       valid_lft forever preferred_lft forever
    inet6 2001:569:7efd:8200:b2b5:a216:77f6:466b/64 scope global dynamic mngtmpaddr noprefixroute 
       valid_lft 7470sec preferred_lft 7170sec
    inet6 fe80::13d6:d1dc:48ad:9dc3/64 scope link 
       valid_lft forever preferred_lft forever

To bring up Wifi is just to plug in/out the Wifi USB dongle.

2.2 Prerequisite Installation

2.2.1 NCSDK

We FIRST need to have ncsdk installed. Yup, here, as described in Run NCS Applications on Raspberry Pi, we carry out the installation directly under folder ...../ncsdk/api/src.

1
2
➜  src git:(ncsdk2) ✗ make -j4
NCSDK FW successfully installed

2.2.2 Caffe

2.2.2.1 Prerequisite Packages

Prerequisite:

For simplicity, you can grab the above three packages directly from Raspbian’s repository. However, it seems the default boost from Raspbian’s repository does NOT support python3 but ONLY python2. Therefore, in my case, I built boost 1.71, flann 1.9.1 and OpenCV 4.1.1 from sractch from source.

2.2.2.2 Cross-compile For flann

The reason why we need to cross-compile flann is because of the limited memory of Raspberry Pi 3B.

In my case, I’m using crosstool-NG and following blog: RPi Linaro GCC Compilation to carry out my cross-compiling. Details about cross-compiling can be found in my next blog Build Toolchain Using Crosstool-NG

Additional Issue: BCM2837 is detected as BCM2835
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
➜  ~ cat /proc/cpuinfo
processor	: 0
model name	: ARMv7 Processor rev 4 (v7l)
BogoMIPS	: 38.40
Features	: half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt vfpd32 lpae evtstrm crc32 
CPU implementer	: 0x41
CPU architecture: 7
CPU variant	: 0x0
CPU part	: 0xd03
CPU revision	: 4

processor	: 1
model name	: ARMv7 Processor rev 4 (v7l)
BogoMIPS	: 38.40
Features	: half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt vfpd32 lpae evtstrm crc32 
CPU implementer	: 0x41
CPU architecture: 7
CPU variant	: 0x0
CPU part	: 0xd03
CPU revision	: 4

processor	: 2
model name	: ARMv7 Processor rev 4 (v7l)
BogoMIPS	: 38.40
Features	: half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt vfpd32 lpae evtstrm crc32 
CPU implementer	: 0x41
CPU architecture: 7
CPU variant	: 0x0
CPU part	: 0xd03
CPU revision	: 4

processor	: 3
model name	: ARMv7 Processor rev 4 (v7l)
BogoMIPS	: 38.40
Features	: half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt vfpd32 lpae evtstrm crc32 
CPU implementer	: 0x41
CPU architecture: 7
CPU variant	: 0x0
CPU part	: 0xd03
CPU revision	: 4

Hardware	: BCM2835
Revision	: a22082
Serial		: 000000003fc1b876

This issue seems to be an OLD topic. More details about the development history of Raspberry Pi can be found on Wikipedia.

2.2.2.3 Build Caffe From Source

Then, we build Caffe 1 from source.

2.3 Test NCS

Now, let’s test the installation of NCS, by the example hello_ncs_py from ncappzoo.

1
2
3
4
➜  hello_ncs_py git:(ncsdk2) python hello_ncs.py 
Hello NCS! Device opened normally.
Goodbye NCS! Device closed normally.
NCS device working.

2.2.3 Test Simple Image Classification

Posted on | Edited on | In

Today, we’re going to build our own toolchain using crosstool-NG. There are so many cases that we want to build a particular toolchain for a specific embedded system development board. One MOST important reason for that is probably because the specific development board has limited resource to build upon, therefore, the time-consumption to build some software on the development board is VERY inefficient. In the other way around, cross compiling on the host PC is MORE efficient with respective to the time-consumption.

In this blog, for simplicity, we take Raspberry Pi 3B as our demo development board, for which we are building the cross compiler. Some references can be found at:

1. Installation

How to install crosstool-NG are thoroughly summarized at its official website. In my case, I had it installed under folder /opt/compilers/crosstool-ng. Let’s take a look:

1
2
3
4
5
6
longervision-GT72-6QE% pwd
/opt/compilers/crosstool-ng
longervision-GT72-6QE% ls
bin  libexec  share
longervision-GT72-6QE% ls bin
ct-ng

And make sure /opt/compilers/crosstool-ng/bin is under environment variable PATH.

2. Configuration

Under any directory that you want to save your .config file, we can configure our target cross compiler.

1
longervision-GT72-6QE% ct-ng menuconfig

Crosstool-NG

According to elinux: RPi Linaro GCC Compilation, we need to do the following selections:

  • Paths and misc options:

    • Try features marked as EXPERIMENTAL: ticked
    • Prefix directory: input the full path where you want to save the built toolchains
    • Number of parallel jobs: 4. Sorry that I’ve NO idea if this is the number of cores on Raspberry Pi 3B, but Raspberry Pi 3B does have 4 cores.

  • Target options:

    • Target Architecture: arm
    • Default instruction set mode: arm
    • Use EABI: ticked
    • append hf to the tuple (EXPERIMENTAL): ticked
    • Endianness: Little endian
    • Bitness: 32-bit
    • Emit assembly for CPU: cortex-a53
    • Use specific FPU: vfp
    • Floating point: hardware (FPU)

  • Toolchain options:

    • Tuple’s vendor string: rpi

  • Operating System

    1
    2
    ➜  ~ uname -a
        Linux raspberrypi 4.19.66-v7+ #1253 SMP Thu Aug 15 11:49:46 BST 2019 armv7l GNU/Linux

  • Binary utilities:

    • Binary format: ELF
    • Version of binutils: 2.31.1
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    ➜  ~ apt show binutils
        Package: binutils
        Version: 2.31.1-16+rpi1
        Priority: optional
        Section: devel
        Maintainer: Matthias Klose <doko@debian.org>
        Installed-Size: 95.2 kB
        Provides: binutils-gold, elf-binutils
        Depends: binutils-common (= 2.31.1-16+rpi1), libbinutils (= 2.31.1-16+rpi1), binutils-arm-linux-gnueabihf (= 2.31.1-16+rpi1)
        Suggests: binutils-doc (>= 2.31.1-16+rpi1)
        Conflicts: binutils-multiarch (<< 2.27-8), modutils (<< 2.4.19-1)
        Homepage: https://www.gnu.org/software/binutils/
        Download-Size: 56.9 kB
        APT-Manual-Installed: no
        APT-Sources: http://raspbian.raspberrypi.org/raspbian buster/main armhf Packages
        Description: GNU assembler, linker and binary utilities
        The programs in this package are used to assemble, link and manipulate
        binary and object files.  They may be used in conjunction with a compiler
        and various libraries to build programs.

  • C-library:

    • C library: glibc
    • Version of glibc: 2.28
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    ➜  ~ apt show libc-bin
        Package: libc-bin
        Version: 2.28-10+rpi1
        Priority: required
        Essential: yes
        Section: libs
        Source: glibc
        Maintainer: GNU Libc Maintainers <debian-glibc@lists.debian.org>
        Installed-Size: 3,015 kB
        Depends: libc6 (>> 2.28), libc6 (<< 2.29)
        Recommends: manpages
        Homepage: https://www.gnu.org/software/libc/libc.html
        Download-Size: 657 kB
        APT-Manual-Installed: yes
        APT-Sources: http://raspbian.raspberrypi.org/raspbian buster/main armhf Packages
        Description: GNU C Library: Binaries
        This package contains utility programs related to the GNU C Library.
        .
        * catchsegv: catch segmentation faults in programs
        * getconf: query system configuration variables
        * getent: get entries from administrative databases
        * iconv, iconvconfig: convert between character encodings
        * ldd, ldconfig: print/configure shared library dependencies
        * locale, localedef: show/generate locale definitions
        * tzselect, zdump, zic: select/dump/compile time zones

  • C compiler:

    • Show gcc versions from: GNU
    • Version of gcc: 8.3.0
    • gcc extra config: --with-float=hard
    • Link libstdc++ statically into the gcc binary: tick
    • C++: tick
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    ➜  ~ apt show gcc
    Package: gcc
    Version: 4:8.3.0-1+rpi2
    Priority: optional
    Section: devel
    Source: gcc-defaults (1.181+rpi2)
    Maintainer: Debian GCC Maintainers <debian-gcc@lists.debian.org>
    Installed-Size: 46.1 kB
    Provides: c-compiler, gcc-arm-linux-gnueabihf (= 4:8.3.0-1+rpi2)
    Depends: cpp (= 4:8.3.0-1+rpi2), gcc-8 (>= 8.3.0-1~)
    Recommends: libc6-dev | libc-dev
    Suggests: gcc-multilib, make, manpages-dev, autoconf, automake, libtool, flex, bison, gdb, gcc-doc
    Conflicts: gcc-doc (<< 1:2.95.3)
    Download-Size: 5,200 B
    APT-Manual-Installed: no
    APT-Sources: http://raspbian.raspberrypi.org/raspbian buster/main armhf Packages
    Description: GNU C compiler
    This is the GNU C compiler, a fairly portable optimizing compiler for C.
    .
    This is a dependency package providing the default GNU C compiler.

After we save the configuration to file .config, we Exit the crosstool-NG configuration dialog.

3. Build

Please remember to:

  • unset LD_LIBRARY_PATH before building. Otherwise, you’ll meet some ERROR messages.
  • mkdir ~/src before building. Otherwise, whenever you tried to rerun ct-ng build, you’ll have to download ALL required packages from scratch.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
longervision-GT72-6QE% ct-ng build
[INFO ]  Performing some trivial sanity checks
[WARN ]  Number of open files 1024 may not be sufficient to build the toolchain; increasing to 2048
[INFO ]  Build started 20190906.014128
[INFO ]  Building environment variables
[EXTRA]  Preparing working directories
[EXTRA]  Installing user-supplied crosstool-NG configuration
[EXTRA]  =================================================================
[EXTRA]  Dumping internal crosstool-NG configuration
[EXTRA]    Building a toolchain for:
[EXTRA]      build  = x86_64-pc-linux-gnu
[EXTRA]      host   = x86_64-pc-linux-gnu
[EXTRA]      target = arm-rpi-linux-gnueabihf
[EXTRA]  Dumping internal crosstool-NG configuration: done in 0.07s (at 00:01)
[INFO ]  =================================================================
[INFO ]  Retrieving needed toolchain components' tarballs
[EXTRA]    Retrieving 'ncurses-6.1'
[EXTRA]    Verifying SHA512 checksum for 'ncurses-6.1.tar.gz'
[EXTRA]    Saving 'ncurses-6.1.tar.gz' to local storage
[EXTRA]    Retrieving 'libiconv-1.15'
[EXTRA]    Verifying SHA512 checksum for 'libiconv-1.15.tar.gz'
[EXTRA]    Saving 'libiconv-1.15.tar.gz' to local storage
[EXTRA]    Retrieving 'gettext-0.19.8.1'
[EXTRA]    Verifying SHA512 checksum for 'gettext-0.19.8.1.tar.xz'
[EXTRA]    Saving 'gettext-0.19.8.1.tar.xz' to local storage
[EXTRA]    Retrieving 'binutils-2.31.1'
[EXTRA]    Verifying SHA512 checksum for 'binutils-2.31.1.tar.xz'
[EXTRA]    Saving 'binutils-2.31.1.tar.xz' to local storage
[EXTRA]    Retrieving 'gcc-8.3.0'
[EXTRA]    Verifying SHA512 checksum for 'gcc-8.3.0.tar.xz'
[EXTRA]    Saving 'gcc-8.3.0.tar.xz' to local storage
[EXTRA]    Retrieving 'glibc-2.28'
[EXTRA]    Verifying SHA512 checksum for 'glibc-2.28.tar.xz'
[EXTRA]    Saving 'glibc-2.28.tar.xz' to local storage
[INFO ]  Retrieving needed toolchain components' tarballs: done in 130.35s (at 02:12)
[INFO ]  =================================================================
[INFO ]  Extracting and patching toolchain components
[EXTRA]    Extracting linux-4.20.8
[EXTRA]    Patching linux-4.20.8
[EXTRA]    Extracting zlib-1.2.11
[EXTRA]    Patching zlib-1.2.11
[EXTRA]    Extracting gmp-6.1.2
[EXTRA]    Patching gmp-6.1.2
[EXTRA]    Extracting mpfr-4.0.2
[EXTRA]    Patching mpfr-4.0.2
[EXTRA]    Extracting isl-0.20
[EXTRA]    Patching isl-0.20
[EXTRA]    Extracting mpc-1.1.0
[EXTRA]    Patching mpc-1.1.0
[EXTRA]    Extracting ncurses-6.1
[EXTRA]    Patching ncurses-6.1
[EXTRA]    Extracting libiconv-1.15
[EXTRA]    Patching libiconv-1.15
[EXTRA]    Extracting gettext-0.19.8.1
[EXTRA]    Patching gettext-0.19.8.1
[EXTRA]    Extracting binutils-2.31.1
[EXTRA]    Patching binutils-2.31.1
[EXTRA]    Extracting gcc-8.3.0
[EXTRA]    Patching gcc-8.3.0
[EXTRA]    Extracting glibc-2.28
[EXTRA]    Patching glibc-2.28
[INFO ]  Extracting and patching toolchain components: done in 33.39s (at 02:45)
[INFO ]  =================================================================
[INFO ]  Installing ncurses for build
[EXTRA]    Configuring ncurses
[EXTRA]    Building ncurses
[EXTRA]    Installing ncurses
[INFO ]  Installing ncurses for build: done in 14.62s (at 03:00)
[INFO ]  =================================================================
[INFO ]  Installing zlib for host
[EXTRA]    Configuring zlib
[EXTRA]    Building zlib
[EXTRA]    Installing zlib
[INFO ]  Installing zlib for host: done in 1.12s (at 03:01)
[INFO ]  =================================================================
[INFO ]  Installing GMP for host
[EXTRA]    Configuring GMP
[EXTRA]    Building GMP
[EXTRA]    Installing GMP
[INFO ]  Installing GMP for host: done in 30.12s (at 03:31)
[INFO ]  =================================================================
[INFO ]  Installing MPFR for host
[EXTRA]    Configuring MPFR
[EXTRA]    Building MPFR
[EXTRA]    Installing MPFR
[INFO ]  Installing MPFR for host: done in 17.35s (at 03:48)
[INFO ]  =================================================================
[INFO ]  Installing ISL for host
[EXTRA]    Configuring ISL
[EXTRA]    Building ISL
[EXTRA]    Installing ISL
[INFO ]  Installing ISL for host: done in 17.73s (at 04:06)
[INFO ]  =================================================================
[INFO ]  Installing MPC for host
[EXTRA]    Configuring MPC
[EXTRA]    Building MPC
[EXTRA]    Installing MPC
[INFO ]  Installing MPC for host: done in 5.47s (at 04:12)
[INFO ]  =================================================================
[INFO ]  Installing ncurses for host
[EXTRA]    Configuring ncurses
[EXTRA]    Building ncurses
[EXTRA]    Installing ncurses
[INFO ]  Installing ncurses for host: done in 13.64s (at 04:25)
[INFO ]  =================================================================
[INFO ]  Installing libiconv for host
[EXTRA]    Skipping (included in GNU C library)
[INFO ]  Installing libiconv for host: done in 0.01s (at 04:25)
[INFO ]  =================================================================
[INFO ]  Installing gettext for host
[EXTRA]    Skipping (included in GNU C library)
[INFO ]  Installing gettext for host: done in 0.01s (at 04:25)
[INFO ]  =================================================================
[INFO ]  Installing binutils for host
[EXTRA]    Configuring binutils
[EXTRA]    Building binutils
[EXTRA]    Installing binutils
[INFO ]  Installing binutils for host: done in 46.89s (at 05:12)
[INFO ]  =================================================================
[INFO ]  Installing pass-1 core C gcc compiler
[EXTRA]    Configuring core C gcc compiler
[EXTRA]    Building gcc
[EXTRA]    Installing gcc
[EXTRA]    Housekeeping for core gcc compiler
[EXTRA]       '' --> lib (gcc)   lib (os)
[INFO ]  Installing pass-1 core C gcc compiler: done in 285.36s (at 09:58)
[INFO ]  =================================================================
[INFO ]  Installing kernel headers
[EXTRA]    Installing kernel headers
[EXTRA]    Checking installed headers
[INFO ]  Installing kernel headers: done in 6.47s (at 10:04)
[INFO ]  =================================================================
[INFO ]  Installing C library headers & start files
[INFO ]    =================================================================
[INFO ]    Building for multilib 1/1: ''
[EXTRA]      Configuring C library
[EXTRA]      Installing C library headers
[EXTRA]      Installing C library start files
[INFO ]    Building for multilib 1/1: '': done in 7.65s (at 10:12)
[INFO ]  Installing C library headers & start files: done in 7.72s (at 10:12)
[INFO ]  =================================================================
[INFO ]  Installing pass-2 core C gcc compiler
[EXTRA]    Configuring core C gcc compiler
[EXTRA]    Building gcc
[EXTRA]    Installing gcc
[EXTRA]    Housekeeping for core gcc compiler
[EXTRA]       '' --> lib (gcc)   lib (os)
[INFO ]  Installing pass-2 core C gcc compiler: done in 354.87s (at 16:07)
[INFO ]  =================================================================
[INFO ]  Installing C library
[INFO ]    =================================================================
[INFO ]    Building for multilib 1/1: ''
[EXTRA]      Cleaning up start files
[EXTRA]      Configuring C library
[EXTRA]      Building C library
[EXTRA]      Configuring C library
[EXTRA]      Building C library
[EXTRA]      Installing C library
[INFO ]    Building for multilib 1/1: '': done in 184.69s (at 19:12)
[INFO ]  Installing C library: done in 184.78s (at 19:12)
[INFO ]  =================================================================
[INFO ]  Installing final gcc compiler
[EXTRA]    Configuring final gcc compiler
[EXTRA]    Building final gcc compiler
[EXTRA]    Installing final gcc compiler
[EXTRA]    Housekeeping for final gcc compiler
[EXTRA]       '' --> lib (gcc)   lib (os)
[INFO ]  Installing final gcc compiler: done in 339.13s (at 24:51)
[INFO ]  =================================================================
[INFO ]  Finalizing the toolchain's directory
[INFO ]    Stripping all toolchain executables
[EXTRA]    Installing the populate helper
[EXTRA]    Installing a cross-ldd helper
[EXTRA]    Creating toolchain aliases
[EXTRA]    Removing installed documentation
[EXTRA]    Collect license information from: ~/Crosscompile/.build/arm-rpi-linux-gnueabihf/src
[EXTRA]    Put the license information to: ~/....../RPi/share/licenses
[INFO ]  Finalizing the toolchain's directory: done in 1.80s (at 24:52)
[INFO ]  Build completed at 20190906.020619
[INFO ]  (elapsed: 24:51.86)
[INFO ]  Finishing installation (may take a few seconds)...
[24:52] / %                                                                                                                                                              longervision-GT72-6QE%

This process may take a while. I’m going to sleep tonight. Continue tomorrow…

Alright, let’s continue today. Glad to know it’s built successfully.

Let’s FIRST take a look at what’s under the current folder.

1
2
longervision-GT72-6QE% ls
build.log  rpi.config

And then, let’s take a look at what’s built under the specified destination folder.

1
2
3
4
5
6
7
8
9
10
longervision-GT72-6QE% ls ~/....../CrossCompile/RPi
arm-rpi-linux-gnueabihf  bin  build.log.bz2  include  lib  libexec  share
longervision-GT72-6QE% ls ~/....../CrossCompile/RPi/bin
arm-rpi-linux-gnueabihf-addr2line  arm-rpi-linux-gnueabihf-cpp           arm-rpi-linux-gnueabihf-gcc-ar      arm-rpi-linux-gnueabihf-gprof    arm-rpi-linux-gnueabihf-objdump   arm-rpi-linux-gnueabihf-strip
arm-rpi-linux-gnueabihf-ar         arm-rpi-linux-gnueabihf-ct-ng.config  arm-rpi-linux-gnueabihf-gcc-nm      arm-rpi-linux-gnueabihf-ld       arm-rpi-linux-gnueabihf-populate
arm-rpi-linux-gnueabihf-as         arm-rpi-linux-gnueabihf-elfedit       arm-rpi-linux-gnueabihf-gcc-ranlib  arm-rpi-linux-gnueabihf-ld.bfd   arm-rpi-linux-gnueabihf-ranlib
arm-rpi-linux-gnueabihf-c++        arm-rpi-linux-gnueabihf-g++           arm-rpi-linux-gnueabihf-gcov        arm-rpi-linux-gnueabihf-ldd      arm-rpi-linux-gnueabihf-readelf
arm-rpi-linux-gnueabihf-cc         arm-rpi-linux-gnueabihf-gcc           arm-rpi-linux-gnueabihf-gcov-dump   arm-rpi-linux-gnueabihf-nm       arm-rpi-linux-gnueabihf-size
arm-rpi-linux-gnueabihf-c++filt    arm-rpi-linux-gnueabihf-gcc-8.3.0     arm-rpi-linux-gnueabihf-gcov-tool   arm-rpi-linux-gnueabihf-objcopy  arm-rpi-linux-gnueabihf-strings
longervision-GT72-6QE%

Finally, let’s take a look at the version of our built cross compilers for Raspberry Pi 3B.

1
2
3
4
5
6
7
8
9
10
11
12
13
longervision-GT72-6QE% arm-rpi-linux-gnueabihf-gcc --version
zsh: command not found: arm-rpi-linux-gnueabihf-gcc
longervision-GT72-6QE% ./arm-rpi-linux-gnueabihf-gcc --version
arm-rpi-linux-gnueabihf-gcc (crosstool-NG 1.24.0.6-afaf7b9) 8.3.0
Copyright (C) 2018 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

longervision-GT72-6QE% ./arm-rpi-linux-gnueabihf-g++ --version
arm-rpi-linux-gnueabihf-g++ (crosstool-NG 1.24.0.6-afaf7b9) 8.3.0
Copyright (C) 2018 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

Additional issue: It seems current crosstool-NG does NOT officially support Python. Please refer to my issue at crosstool-NG

4. Compile/Build a Package with Generated Toolchain

For most of the packages nowadays, they are

  • either supported by make: ./configure -> make -> make install
  • or supported by CMake: mkdir build -> cd build -> ccmake ../ -> make -> make install

How to use our generated toolchain to compile/build our target packages?

Today, we’re going to take the package flann as an example, which are to be built with CMake.

4.1 CMakeLists.txt Modification

4.1.1 CMake Toolchains - Cross Compiling for Linux

By following CMake Toolchains - Cross Compiling for Linux, we FIRST modify flann CMakeLists.txt a bit by adding the following lines before project(flann).

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
cmake_minimum_required(VERSION 2.6)

if(COMMAND cmake_policy)
    cmake_policy(SET CMP0003 NEW)
endif(COMMAND cmake_policy)

set(CMAKE_SYSTEM_NAME Linux)
set(CMAKE_SYSTEM_PROCESSOR arm)

set(CMAKE_SYSROOT ....../arm-rpi-linux-gnueabihf/sysroot)
set(CMAKE_STAGING_PREFIX ....../stage)

set(tools /opt/compilers/RPi)
set(CMAKE_C_COMPILER ${tools}/bin/arm-rpi-linux-gnueabihf-gcc)
set(CMAKE_CXX_COMPILER ${tools}/bin/arm-rpi-linux-gnueabihf-g++)

set(CMAKE_FIND_ROOT_PATH_MODE_PROGRAM NEVER)
set(CMAKE_FIND_ROOT_PATH_MODE_LIBRARY ONLY)
set(CMAKE_FIND_ROOT_PATH_MODE_INCLUDE ONLY)
set(CMAKE_FIND_ROOT_PATH_MODE_PACKAGE ONLY)

set(CMAKE_TRY_COMPILE_TARGET_TYPE "STATIC_LIBRARY")

project(flann)
string(TOLOWER ${PROJECT_NAME} PROJECT_NAME_LOWER)
  • CMAKE_SYSROOT specify the sysroot directory which emulate your target environment, here, Raspberry Pi 3B
  • CMAKE_STAGING_PREFIX is where the STAGING results store, for the reason that final results may require to be built in multiple stages. You may refer to Linux From Scratch for further background knowledge about that.
  • tools actually specify the building tool directory. Making sure all generated cross compiling tools are under folder ${tools}/bin.

4.1.2 Ignore hdf5

In addition, for the emulated Raspberry Pi 3B sysroot, hdf5 is NOT supported. Therefore, let’s simply comment out the following line in flann CMakeLists.txt.

1
#find_hdf5()

4.2 Cross Compile

Now, let’s start cross-compiling flann.

1
2
3
longervision-GT72-6QE% mkdir build
longervision-GT72-6QE% cd build
longervision-GT72-6QE% ccmake ../

and press c and then t, you’ll see the cross compiling toolchains have been automatically configured as:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
BUILD_CUDA_LIB                  *OFF
BUILD_C_BINDINGS                *ON
BUILD_DOC                       *OFF
BUILD_EXAMPLES                  *OFF
BUILD_MATLAB_BINDINGS           *OFF
BUILD_PYTHON_BINDINGS           *ON
BUILD_TESTS                     *OFF
CMAKE_AR                        */opt/compilers/RPi/bin/arm-rpi-linux-gnueabihf-ar
CMAKE_BUILD_TYPE                *RelWithDebInfo
CMAKE_COLOR_MAKEFILE            *ON
CMAKE_CXX_COMPILER_AR           */opt/compilers/RPi/bin/arm-rpi-linux-gnueabihf-gcc-ar
CMAKE_CXX_COMPILER_RANLIB       */opt/compilers/RPi/bin/arm-rpi-linux-gnueabihf-gcc-ranlib
CMAKE_CXX_FLAGS                 *-I/usr/include
CMAKE_CXX_FLAGS_DEBUG           *-g
CMAKE_CXX_FLAGS_MINSIZEREL      *-Os -DNDEBUG
CMAKE_CXX_FLAGS_RELEASE         *-O3 -DNDEBUG
CMAKE_CXX_FLAGS_RELWITHDEBINFO  *-O2 -g -DNDEBUG
CMAKE_C_COMPILER_AR             */opt/compilers/RPi/bin/arm-rpi-linux-gnueabihf-gcc-ar
CMAKE_C_COMPILER_RANLIB         */opt/compilers/RPi/bin/arm-rpi-linux-gnueabihf-gcc-ranlib
CMAKE_C_FLAGS                   *-I/usr/include
CMAKE_C_FLAGS_DEBUG             *-g
CMAKE_C_FLAGS_MINSIZEREL        *-Os -DNDEBUG
CMAKE_C_FLAGS_RELEASE           *-O3 -DNDEBUG
CMAKE_C_FLAGS_RELWITHDEBINFO    *-O2 -g -DNDEBUG
CMAKE_EXE_LINKER_FLAGS          *
CMAKE_EXE_LINKER_FLAGS_DEBUG    *
CMAKE_EXE_LINKER_FLAGS_MINSIZE  *  
CMAKE_EXE_LINKER_FLAGS_RELEASE  *     
CMAKE_EXE_LINKER_FLAGS_RELWITH  *
CMAKE_EXPORT_COMPILE_COMMANDS   *OFF
CMAKE_INSTALL_PREFIX            */usr/local
CMAKE_LINKER                    */opt/compilers/RPi/bin/arm-rpi-linux-gnueabihf-ld
CMAKE_MAKE_PROGRAM              */usr/bin/make
CMAKE_MODULE_LINKER_FLAGS       *    
CMAKE_MODULE_LINKER_FLAGS_DEBU  *  
CMAKE_MODULE_LINKER_FLAGS_MINS  *   
CMAKE_MODULE_LINKER_FLAGS_RELE  *    
CMAKE_MODULE_LINKER_FLAGS_RELW  * 
CMAKE_NM                        */opt/compilers/RPi/bin/arm-rpi-linux-gnueabihf-nm
CMAKE_OBJCOPY                   */opt/compilers/RPi/bin/arm-rpi-linux-gnueabihf-objcopy
CMAKE_OBJDUMP                   */opt/compilers/RPi/bin/arm-rpi-linux-gnueabihf-objdump
CMAKE_RANLIB                    */opt/compilers/RPi/bin/arm-rpi-linux-gnueabihf-ranlib
CMAKE_SHARED_LINKER_FLAGS       *    
CMAKE_SHARED_LINKER_FLAGS_DEBU  *
CMAKE_SHARED_LINKER_FLAGS_MINS  *   
CMAKE_SHARED_LINKER_FLAGS_RELE  *
CMAKE_SHARED_LINKER_FLAGS_RELW  *   
CMAKE_SKIP_INSTALL_RPATH        *OFF
CMAKE_SKIP_RPATH                *OFF
CMAKE_STATIC_LINKER_FLAGS       *
CMAKE_STATIC_LINKER_FLAGS_DEBU  *
CMAKE_STATIC_LINKER_FLAGS_MINS  * 
CMAKE_STATIC_LINKER_FLAGS_RELE  *
CMAKE_STATIC_LINKER_FLAGS_RELW  *
CMAKE_STRIP                     */opt/compilers/RPi/bin/arm-rpi-linux-gnueabihf-strip
CMAKE_VERBOSE_MAKEFILE          *ON
CPACK_SOURCE_RPM                *OFF
CPACK_SOURCE_TBZ2               *ON
CPACK_SOURCE_TGZ                *ON
CPACK_SOURCE_TXZ                *ON 
CPACK_SOURCE_TZ                 *ON
CPACK_SOURCE_ZIP                *OFF
DPKG_PROGRAM                    */usr/bin/dpkg
NSIS_PROGRAM                    */usr/bin/makensis
NVCC_COMPILER_BINDIR            *
PACKAGE_MAKER_PROGRAM           *PACKAGE_MAKER_PROGRAM-NOTFOUND
PKG_CONFIG_EXECUTABLE           */usr/bin/pkg-config
PYTHON_EXECUTABLE               */usr/bin/python
RPM_PROGRAM                     *RPM_PROGRAM-NOTFOUND
USE_MPI                         *OFF
USE_OPENMP                      *OFF

The LAST step before make is to modify some of the parameters accordingly, in my case:

  • BUILD_PYTHON_BINDINGS: ON -> OFF
  • CMAKE_CXX_FLAGS: -I/usr/include (for lz4.h from my host Ubuntu 19.04)
  • CMAKE_C_FLAGS: -I/usr/include (for lz4.h from my host Ubuntu 19.04)
  • CMAKE_VERBOSE_MAKEFILE: OFF -> ON
  • CMAKE_INSTALL_PREFIX: /usr/local/ -> under which directory you want to install, which can be IGNORED for now

Now, press c, you’ll see

1
2
CMake Warning at CMakeLists.txt:99 (message):
  hdf5 library not found, some tests will not be run

A warining of missing hdf5 is clearly reasonable and acceptable. Then press g.

Finally, it’s the time for us to cross build flann.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
longervision-GT72-6QE% make -j8
/usr/bin/cmake -S....../flann -B....../flann/build --check-build-system CMakeFiles/Makefile.cmake 0
/usr/bin/cmake -E cmake_progress_start ....../flann/build/CMakeFiles ....../flann/build/CMakeFiles/progress.marks
make -f CMakeFiles/Makefile2 all
make[1]: Entering directory '....../flann/build'
make -f src/cpp/CMakeFiles/flann_s.dir/build.make src/cpp/CMakeFiles/flann_s.dir/depend
make -f src/cpp/CMakeFiles/flann_cpp_s.dir/build.make src/cpp/CMakeFiles/flann_cpp_s.dir/depend
make[2]: Entering directory '....../flann/build'
cd ....../flann/build && /usr/bin/cmake -E cmake_depends "Unix Makefiles" ....../flann ....../flann/src/cpp ....../flann/build ....../flann/build/src/cpp ....../flann/build/src/cpp/CMakeFiles/flann_s.dir/DependInfo.cmake --color=
make[2]: Entering directory '....../flann/build'
cd ....../flann/build && /usr/bin/cmake -E cmake_depends "Unix Makefiles" ....../flann ....../flann/src/cpp ....../flann/build ....../flann/build/src/cpp ....../flann/build/src/cpp/CMakeFiles/flann_cpp_s.dir/DependInfo.cmake --color=
Scanning dependencies of target flann_cpp_s
make[2]: Leaving directory '....../flann/build'
make -f src/cpp/CMakeFiles/flann_s.dir/build.make src/cpp/CMakeFiles/flann_s.dir/build
make[2]: Entering directory '....../flann/build'
make[2]: Leaving directory '....../flann/build'
make -f src/cpp/CMakeFiles/flann_cpp_s.dir/build.make src/cpp/CMakeFiles/flann_cpp_s.dir/build
make[2]: Entering directory '....../flann/build'
[ 12%] Building CXX object src/cpp/CMakeFiles/flann_s.dir/flann/flann.cpp.o
cd ....../flann/build/src/cpp && /opt/compilers/RPi/bin/arm-rpi-linux-gnueabihf-g++ --sysroot=....../RPi/arm-rpi-linux-gnueabihf/sysroot  -DFLANN_STATIC -D_FLANN_VERSION=1.9.1 -I....../flann/src/cpp  -I/usr/include -O2 -g -DNDEBUG   -Wall -Wno-unknown-pragmas -Wno-unused-function -fPIC -o CMakeFiles/flann_s.dir/flann/flann.cpp.o -c ....../flann/src/cpp/flann/flann.cpp
[ 25%] Building CXX object src/cpp/CMakeFiles/flann_cpp_s.dir/flann/flann_cpp.cpp.o
cd ....../flann/build/src/cpp && /opt/compilers/RPi/bin/arm-rpi-linux-gnueabihf-g++ --sysroot=....../RPi/arm-rpi-linux-gnueabihf/sysroot  -DFLANN_STATIC -DFLANN_USE_CUDA -D_FLANN_VERSION=1.9.1 -I....../flann/src/cpp  -I/usr/include -O2 -g -DNDEBUG   -Wall -Wno-unknown-pragmas -Wno-unused-function -fPIC -o CMakeFiles/flann_cpp_s.dir/flann/flann_cpp.cpp.o -c ....../flann/src/cpp/flann/flann_cpp.cpp
In file included from ....../flann/src/cpp/flann/util/params.h:33,
                 from ....../flann/src/cpp/flann/flann.hpp:42,
                 from ....../flann/src/cpp/flann/flann.h:603,
                 from ....../flann/src/cpp/flann/flann.cpp:31:
....../flann/src/cpp/flann/util/any.h: In member function 'virtual void flann::anyimpl::big_any_policy<T>::static_delete(void**)':
....../flann/src/cpp/flann/util/any.h:81:9: warning: this 'if' clause does not guard... [-Wmisleading-indentation]
         if (* x) delete (* reinterpret_cast<T**>(x)); *x = NULL;
         ^~
....../flann/src/cpp/flann/util/any.h:81:55: note: ...this statement, but the latter is misleadingly indented as if it were guarded by the 'if'
         if (* x) delete (* reinterpret_cast<T**>(x)); *x = NULL;
                                                       ^
In file included from ....../flann/src/cpp/flann/util/params.h:33,
                 from ....../flann/src/cpp/flann/flann.hpp:42,
                 from ....../flann/src/cpp/flann/flann_cpp.cpp:30:
....../flann/src/cpp/flann/util/any.h: In member function 'virtual void flann::anyimpl::big_any_policy<T>::static_delete(void**)':
....../flann/src/cpp/flann/util/any.h:81:9: warning: this 'if' clause does not guard... [-Wmisleading-indentation]
         if (* x) delete (* reinterpret_cast<T**>(x)); *x = NULL;
         ^~
....../flann/src/cpp/flann/util/any.h:81:55: note: ...this statement, but the latter is misleadingly indented as if it were guarded by the 'if'
         if (* x) delete (* reinterpret_cast<T**>(x)); *x = NULL;
                                                       ^
[ 37%] Linking CXX static library ../../lib/libflann_cpp_s.a
cd ....../flann/build/src/cpp && /usr/bin/cmake -P CMakeFiles/flann_cpp_s.dir/cmake_clean_target.cmake
cd ....../flann/build/src/cpp && /usr/bin/cmake -E cmake_link_script CMakeFiles/flann_cpp_s.dir/link.txt --verbose=1
/opt/compilers/RPi/bin/arm-rpi-linux-gnueabihf-ar qc ../../lib/libflann_cpp_s.a  CMakeFiles/flann_cpp_s.dir/flann/flann_cpp.cpp.o
/opt/compilers/RPi/bin/arm-rpi-linux-gnueabihf-ranlib ../../lib/libflann_cpp_s.a
make[2]: Leaving directory '....../flann/build'
[ 37%] Built target flann_cpp_s
make -f src/cpp/CMakeFiles/flann_cpp.dir/build.make src/cpp/CMakeFiles/flann_cpp.dir/depend
make[2]: Entering directory '....../flann/build'
cd ....../flann/build && /usr/bin/cmake -E cmake_depends "Unix Makefiles" ....../flann ....../flann/src/cpp ....../flann/build ....../flann/build/src/cpp ....../flann/build/src/cpp/CMakeFiles/flann_cpp.dir/DependInfo.cmake --color=
Scanning dependencies of target flann_cpp
make[2]: Leaving directory '....../flann/build'
make -f src/cpp/CMakeFiles/flann_cpp.dir/build.make src/cpp/CMakeFiles/flann_cpp.dir/build
make[2]: Entering directory '....../flann/build'
[ 50%] Building CXX object src/cpp/CMakeFiles/flann_cpp.dir/empty.cpp.o
cd ....../flann/build/src/cpp && /opt/compilers/RPi/bin/arm-rpi-linux-gnueabihf-g++ --sysroot=....../RPi/arm-rpi-linux-gnueabihf/sysroot  -DFLANN_EXPORTS -D_FLANN_VERSION=1.9.1 -I....../flann/src/cpp  -I/usr/include -O2 -g -DNDEBUG -fPIC   -Wall -Wno-unknown-pragmas -Wno-unused-function -o CMakeFiles/flann_cpp.dir/empty.cpp.o -c ....../flann/src/cpp/empty.cpp
[ 62%] Linking CXX shared library ../../lib/libflann_cpp.so
cd ....../flann/build/src/cpp && /usr/bin/cmake -E cmake_link_script CMakeFiles/flann_cpp.dir/link.txt --verbose=1
/opt/compilers/RPi/bin/arm-rpi-linux-gnueabihf-g++ --sysroot=....../RPi/arm-rpi-linux-gnueabihf/sysroot -fPIC -I/usr/include -O2 -g -DNDEBUG  -shared -Wl,-soname,libflann_cpp.so.1.9 -o ../../lib/libflann_cpp.so.1.9.1 CMakeFiles/flann_cpp.dir/empty.cpp.o -Wl,-whole-archive ../../lib/libflann_cpp_s.a -Wl,-no-whole-archive 
cd ....../flann/build/src/cpp && /usr/bin/cmake -E cmake_symlink_library ../../lib/libflann_cpp.so.1.9.1 ../../lib/libflann_cpp.so.1.9 ../../lib/libflann_cpp.so
make[2]: Leaving directory '....../flann/build'
[ 62%] Built target flann_cpp
In file included from ....../flann/src/cpp/flann/algorithms/all_indices.h:35,
                 from ....../flann/src/cpp/flann/flann.hpp:45,
                 from ....../flann/src/cpp/flann/flann.h:603,
                 from ....../flann/src/cpp/flann/flann.cpp:31:

VERY Important:

  • If you build flann from source on a Raspberry Pi 3B, your system is going to hang HERE, possibly due to lack of memory..
  • Raspberry Pi 3B has 4 cores ONLY, but now you can use MORE cores on your host PC, which is clearly one advantage of using cross compiling.

Finally, you’ll see flann has been successfully cross compiled as:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
....../flann/src/cpp/flann/algorithms/autotuned_index.h: In member function 'void flann::AutotunedIndex<Distance>::optimizeKMeans(std::vector<flann::AutotunedIndex<Distance>::CostData>&) [with Distance = flann::L2<double>]':
....../flann/src/cpp/flann/algorithms/autotuned_index.h:379:31: note: parameter passing for argument of type 'flann::L2<double>' changed in GCC 7.1
         KMeansIndex<Distance> kmeans(sampledDataset_, cost.params, distance_);
                               ^~~~~~
[ 75%] Linking CXX static library ../../lib/libflann_s.a
cd ....../flann/build/src/cpp && /usr/bin/cmake -P CMakeFiles/flann_s.dir/cmake_clean_target.cmake
cd ....../flann/build/src/cpp && /usr/bin/cmake -E cmake_link_script CMakeFiles/flann_s.dir/link.txt --verbose=1
/opt/compilers/RPi/bin/arm-rpi-linux-gnueabihf-ar qc ../../lib/libflann_s.a  CMakeFiles/flann_s.dir/flann/flann.cpp.o
/opt/compilers/RPi/bin/arm-rpi-linux-gnueabihf-ranlib ../../lib/libflann_s.a
make[2]: Leaving directory '....../flann/build'
[ 75%] Built target flann_s
make -f src/cpp/CMakeFiles/flann.dir/build.make src/cpp/CMakeFiles/flann.dir/depend
make[2]: Entering directory '....../flann/build'
cd ....../flann/build && /usr/bin/cmake -E cmake_depends "Unix Makefiles" ....../flann ....../flann/src/cpp ....../flann/build ....../flann/build/src/cpp ....../flann/build/src/cpp/CMakeFiles/flann.dir/DependInfo.cmake --color=
Scanning dependencies of target flann
make[2]: Leaving directory '....../flann/build'
make -f src/cpp/CMakeFiles/flann.dir/build.make src/cpp/CMakeFiles/flann.dir/build
make[2]: Entering directory '....../flann/build'
[ 87%] Building CXX object src/cpp/CMakeFiles/flann.dir/empty.cpp.o
cd ....../flann/build/src/cpp && /opt/compilers/RPi/bin/arm-rpi-linux-gnueabihf-g++ --sysroot=....../RPi/arm-rpi-linux-gnueabihf/sysroot  -DFLANN_EXPORTS -D_FLANN_VERSION=1.9.1 -I....../flann/src/cpp  -I/usr/include -O2 -g -DNDEBUG -fPIC   -Wall -Wno-unknown-pragmas -Wno-unused-function -o CMakeFiles/flann.dir/empty.cpp.o -c ....../flann/src/cpp/empty.cpp
[100%] Linking CXX shared library ../../lib/libflann.so
cd ....../flann/build/src/cpp && /usr/bin/cmake -E cmake_link_script CMakeFiles/flann.dir/link.txt --verbose=1
/opt/compilers/RPi/bin/arm-rpi-linux-gnueabihf-g++ --sysroot=....../RPi/arm-rpi-linux-gnueabihf/sysroot -fPIC -I/usr/include -O2 -g -DNDEBUG  -shared -Wl,-soname,libflann.so.1.9 -o ../../lib/libflann.so.1.9.1 CMakeFiles/flann.dir/empty.cpp.o -Wl,-whole-archive ../../lib/libflann_s.a -Wl,-no-whole-archive -Wl,--push-state,--no-as-needed -Wl,--pop-state 
cd ....../flann/build/src/cpp && /usr/bin/cmake -E cmake_symlink_library ../../lib/libflann.so.1.9.1 ../../lib/libflann.so.1.9 ../../lib/libflann.so
make[2]: Leaving directory '....../flann/build'
[100%] Built target flann
make[1]: Leaving directory '....../flann/build'
/usr/bin/cmake -E cmake_progress_start ....../flann/build/CMakeFiles 0

You can now:

  • make install to install the built/generated libraries installed under CMAKE_INSTALL_PREFIX
  • copy and paste the built/generated libraries onto Raspberry Pi 3B and use it directly.

BTW: do NOT forget to install the header files.

5. Additional Issues

Multilib/multiarch seems to be problematic nowadays. Please pay attention to Multilib/multiarch. Some related issues are enumuated as the end of this blog.

Alright, that’s all for today. Let me go to bed. Good bye…

Posted on | Edited on | In ,

Today, we are going to talk about a fabulous project: stereo vision on a zynq-7010 board.

1. VCSBC nano Z-RH-2

1.1 Hardware

We are using a VCSBC nano Z-RH-2 board for today’s experiment. The board adopted looks like the following:

Front Back Connector
Front Back Connector

More detailed specifications, please refer to Vision Components’s official website.

1.2 Software

After you set up a static IP for this Vision Components SBC, it’s pretty straightforward to ssh into the system.

1
2
3
4
5
6
7
8
9
10
11
longervision-GT72-6QE% ssh user@192.168.1.79
user@192.168.1.79's password: 

The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.
Last login: Thu Aug 29 18:17:49 2019 from 192.168.1.200
user@VC-Z:~$

Currently, VC provides Linux Kernel 3.14.79.

1
2
user@VC-Z:~$ uname -a
Linux VC-Z 3.14.79-vc-z #2 SMP PREEMPT Wed Apr 24 18:33:06 CEST 2019 armv7l GNU/Linux

And, let’s take a look at the dual ARMv7 CPUs on zynq-7010.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
user@VC-Z:~$ cat /proc/cpuinfo
processor	: 0
model name	: ARMv7 Processor rev 0 (v7l)
BogoMIPS	: 1725.23
Features	: swp half thumb fastmult vfp edsp neon vfpv3 tls vfpd32 
CPU implementer	: 0x41
CPU architecture: 7
CPU variant	: 0x3
CPU part	: 0xc09
CPU revision	: 0

processor	: 1
model name	: ARMv7 Processor rev 0 (v7l)
BogoMIPS	: 1725.23
Features	: swp half thumb fastmult vfp edsp neon vfpv3 tls vfpd32 
CPU implementer	: 0x41
CPU architecture: 7
CPU variant	: 0x3
CPU part	: 0xc09
CPU revision	: 0

Hardware	: Xilinx Zynq Platform
Revision	: 0000
Serial		: 0000000000000000

2. Stereo Vision

Sorry everybody. Today, I ONLY test stereo vision on ARM. I’ll try to figure out how to flash an open source IP of stereo vision onto zynq-7010, or write my own ASAP.

Hmmmmmmmm… It’s better I keep my code in dark???

2.1 Classical Image Pairs

Let’s try out the stereo vision on some .pgm image pairs FIRST.

1
2
3
4
5
user@VC-Z:~/longervision$ ./pgmpair ../images/aloe_left.pgm ../images/aloe_right.pgm 
width= 1282
height= 1110
fps: 0.119696
process: 398651

My GOD… It’s UNBELIEVABLY SLOW.

aloe_left aloe_right
aloe_left aloe_right
aloe_left Stereo aloe_right Stereo
aloe_left stereo aloe_right stereo

2.2 Live Video Pairs

The BEST demo code to test Vision Componenets Stereo Vision is Eclipse_Example_Projects_VC_Z.

2.2.1 imageCaptureTest

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
#./imageCaptureTest 
0


 ----------------------------------------------------------------------- 
  VCCapt @0xbe87153c                                    Sensor Count: 2  
 ----------------------------------------------------------------------- 
                                                                         
                  Camera Type :  NANO Z RH Rev.0                         
                Sensor 0 Type :  E2V_EV76C560ABT_1_0                     
                Sensor 1 Type :  E2V_EV76C560ABT_1_0                     
                                                                         
   Trigger        Source      :  IMM                                     
                                                                         
     ___________________________________________________________________ 
     Sensor[ 0]:         Shutter:     12345 us,  Gain: 0.00              
                                                                         
                   x0, y0       :  (    0,    0)                         
            binX,     binY      :  (    1,    1)                         
        subsmplX, subsmplY, flip:  (    1,    1,0x00)                    
    (img)          dx, dy, pitch:  ( 1280, 1024), 1280                   
                                                                         
    (img) st, ccmp1, ccmp2, type:  0xb6b03080, 0x00000000, 0x00000000, 0 
                                                                         
      Description                                                        
       Sen dx, Sen dy, Sen pitch:   1280, 1024, 1280                     
       Max dx, Max dy, Max pitch:   1280, 1024, 1280                     
    Binning Max (dx, dy), Square:      2,    1,    1                     
    Subsmpl Max (dx, dy), Square:      8,    8,    0                     
       Init       st,ccmp1,ccmp2:  0xb6b03080, 0x00000000, 0x00000000    
       Init  Shutter, Gain, Type:       10000,  0.00, GREY   (0)         
                  FD, Port, Type:  4, 0, E2V_EV76C560ABT_1_0             
                                                                         
      Volatile Description                                               
            requestCaptID, State:          0, IDLE                       
          captId, TimestampTicks:          0,   6476832867277            
                                                                         
      Externals Description                                              
            pos[0-3], angle[0-2]:  [ 0.00, 0.00, 0.00],[  0.0,  0.0]     
                                                                         
     ___________________________________________________________________ 
     Sensor[ 1]:         Shutter:     10000 us,  Gain: 0.00              
                                                                         
                   x0, y0       :  (    0,    0)                         
            binX,     binY      :  (    1,    1)                         
        subsmplX, subsmplY, flip:  (    1,    1,0x00)                    
    (img)          dx, dy, pitch:  ( 1280, 1024), 1280                   
                                                                         
    (img) st, ccmp1, ccmp2, type:  0xb69c2080, 0x00000000, 0x00000000, 0 
                                                                         
      Description                                                        
       Sen dx, Sen dy, Sen pitch:   1280, 1024, 1280                     
       Max dx, Max dy, Max pitch:   1280, 1024, 1280                     
    Binning Max (dx, dy), Square:      2,    1,    1                     
    Subsmpl Max (dx, dy), Square:      8,    8,    0                     
       Init       st,ccmp1,ccmp2:  0xb69c2080, 0x00000000, 0x00000000    
       Init  Shutter, Gain, Type:       10000,  0.00, GREY   (0)         
                  FD, Port, Type:  5, 1, E2V_EV76C560ABT_1_0             
                                                                         
      Volatile Description                                               
            requestCaptID, State:          0, IDLE                       
          captId, TimestampTicks:          0,   6476832867284            
                                                                         
      Externals Description                                              
            pos[0-3], angle[0-2]:  [ 0.00, 0.00, 0.00],[  0.0,  0.0]     
                                                                         
 ----------------------------------------------------------------------- 
......

2.2.2 imageCaptFPS

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
03:41:47[root@VC-Z] /home/user/vc/Eclipse_Example_Projects_VC_Z/imageCaptFPS
#./imageCaptFPS 
dy= 1024  Clock=100000000  Acquisiton&Copy Duration:         28s     780685us  for 1000 Cycles ==  34.745525fps.

dy= 768  Clock=100000000  Acquisiton&Copy Duration:         28s     789899us  for 1000 Cycles ==  34.734402fps.

dy= 640  Clock=100000000  Acquisiton&Copy Duration:         28s     790000us  for 1000 Cycles ==  34.734283fps.

dy= 512  Clock=100000000  Acquisiton&Copy Duration:         28s     790012us  for 1000 Cycles ==  34.734268fps.

dy= 384  Clock=100000000  Acquisiton&Copy Duration:         28s     790005us  for 1000 Cycles ==  34.734280fps.

dy= 256  Clock=100000000  Acquisiton&Copy Duration:         28s     790000us  for 1000 Cycles ==  34.734283fps.

dy= 192  Clock=100000000  Acquisiton&Copy Duration:         28s     790021us  for 1000 Cycles ==  34.734257fps.

dy= 128  Clock=100000000  Acquisiton&Copy Duration:         28s     789984us  for 1000 Cycles ==  34.734303fps.

dy= 64  Clock=100000000  Acquisiton&Copy Duration:         28s     789987us  for 1000 Cycles ==  34.734299fps.

dy= 32  Clock=100000000  Acquisiton&Copy Duration:         28s     789997us  for 1000 Cycles ==  34.734287fps.

dy= 16  Clock=100000000  Acquisiton&Copy Duration:         28s     790017us  for 1000 Cycles ==  34.734264fps.

dy= 8  Clock=100000000  Acquisiton&Copy Duration:         28s     790002us  for 1000 Cycles ==  34.734280fps.

dy= 4  Clock=100000000  Acquisiton&Copy Duration:         28s     789985us  for 1000 Cycles ==  34.734303fps.

dy= 2  Clock=100000000  Acquisiton&Copy Duration:         28s     789997us  for 1000 Cycles ==  34.734287fps.

dy= 1  Clock=100000000  Acquisiton&Copy Duration:         28s     789994us  for 1000 Cycles ==  34.734291fps.

The above 2 examples are directly run on Vision Components’s board without display, for the given cable is of VGA connector, which is ALREADY outdated for many years. Therefore, in order to show the captured image pairs, in the next section, we’ll have to stream the captured data to a host computer, and display the real-time video pairs.

2.3 Stream/Display From Host Computer

Due to Vision Componenets’ Official documentation VCLinux_Getting_Started.pdf, images captured from camera can be displayed on a remote hosting PC.

2.3.1 Eclipse_Example_Projects_VC_Z (Not Preferred)

Eclipse_Example_Projects_VC_Z.zip provides some source code, which displays images captured from camera on the hosting PC’s Eclipse, by adding the camera as a Remote System to Eclipse.
Anyway, by using this method, you need to prepare ALL the following software and packages in advance.

2.3.2 vcimgnetclient.py

Besides the above method, a much more straightforward method is to adopt python script vcimgnetclient.py provided by Vision Components. However, vcimgnetclient.py is ONLY python2 compatible, and Vision Components has NO plan to provide a python3 compatible version of vcimgnetclient.py.

Therefore, the KEY to use the 2nd method is to make vcimgnetclient.py python3 compatible.

2.3.2.1 PyGTK to PyGTK3

1) 2to3
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
longervision-GT72-6QE% 2to3 vcimgnetclient.py
RefactoringTool: Skipping optional fixer: buffer
RefactoringTool: Skipping optional fixer: idioms
RefactoringTool: Skipping optional fixer: set_literal
RefactoringTool: Skipping optional fixer: ws_comma
RefactoringTool: Refactored vcimgnetclient.py
--- vcimgnetclient.py	(original)
+++ vcimgnetclient.py	(refactored)
@@ -274,11 +274,11 @@
             if(chk_rgb_true):
                 dy /= 3
                 #img_data_packed = bytearray(chain.from_iterable(izip(mv_img_data[2 * pitch * dy:], mv_img_data[1 * pitch * dy:], mv_img_data[0 * pitch * dy:])))
-                img_data_packed = bytearray(chain.from_iterable(zip(mv_img_data[2 * pitch * dy:], mv_img_data[1 * pitch * dy:], mv_img_data[0 * pitch * dy:])))
+                img_data_packed = bytearray(chain.from_iterable(list(zip(mv_img_data[2 * pitch * dy:], mv_img_data[1 * pitch * dy:], mv_img_data[0 * pitch * dy:]))))
             else:
                 dy /= 1
                 #img_data_packed = bytearray(chain.from_iterable(izip(mv_img_data[0 * pitch * dy:], mv_img_data[0 * pitch * dy:], mv_img_data[0 * pitch * dy:])))
-            img_data_packed = bytearray(chain.from_iterable(zip(mv_img_data[0 * pitch * dy:], mv_img_data[0 * pitch * dy:], mv_img_data[0 * pitch * dy:])))
+            img_data_packed = bytearray(chain.from_iterable(list(zip(mv_img_data[0 * pitch * dy:], mv_img_data[0 * pitch * dy:], mv_img_data[0 * pitch * dy:]))))
             cpb = Gdk.pixbuf_new_from_data(buffer(img_data_packed), Gdk.COLORSPACE_RGB, False, 8, dx, dy, 3*pitch)
             cpb.save(path, "png")
             
@@ -589,16 +589,16 @@
                         dy /= 3
                         mv_img_data = memoryview(img['data'])
                         #img_data_packed = bytearray(chain.from_iterable(izip(mv_img_data[2 * pitch * dy:], mv_img_data[1 * pitch * dy:], mv_img_data[0 * pitch * dy:])))
-                        img_data_packed = bytearray(chain.from_iterable(zip(mv_img_data[2 * pitch * dy:], mv_img_data[1 * pitch * dy:], mv_img_data[0 * pitch * dy:])))
+                        img_data_packed = bytearray(chain.from_iterable(list(zip(mv_img_data[2 * pitch * dy:], mv_img_data[1 * pitch * dy:], mv_img_data[0 * pitch * dy:]))))
                         self.area.window.draw_rgb_image(gc, 0,0, dx,dy, Gdk.RGB_DITHER_NONE, buffer(img_data_packed),3*pitch,0,0)
                     else:
                         if('flscol'==self.how_select):
                             mv_img_data = memoryview(img['data']).tolist()
-                            r_data = map(falsecolor_r,mv_img_data[0 * pitch * dy:])
-                            g_data = map(falsecolor_g,mv_img_data[0 * pitch * dy:])
-                            b_data = map(falsecolor_b,mv_img_data[0 * pitch * dy:])
+                            r_data = list(map(falsecolor_r,mv_img_data[0 * pitch * dy:]))
+                            g_data = list(map(falsecolor_g,mv_img_data[0 * pitch * dy:]))
+                            b_data = list(map(falsecolor_b,mv_img_data[0 * pitch * dy:]))
                             #img_data_packed = bytearray(chain.from_iterable(izip(r_data, g_data, b_data)))
-                            img_data_packed = bytearray(chain.from_iterable(zip(r_data, g_data, b_data)))
+                            img_data_packed = bytearray(chain.from_iterable(list(zip(r_data, g_data, b_data))))
                             self.area.window.draw_rgb_image(gc, 0,0, dx,dy, Gdk.RGB_DITHER_NONE, buffer(img_data_packed),3*pitch,0,0)
                         else: #grey
                             self.area.window.draw_gray_image(gc, 0,0, 
RefactoringTool: Files that need to be modified:
RefactoringTool: vcimgnetclient.py
2) pygi-convert

Please refer to General Porting Tips

1
2
longervision-GT72-6QE% ./pygi-convert.sh vcimgnetclient.py
longervision-GT72-6QE%
3) Try Running
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
longervision-GT72-6QE% python vcimgnetclient.py 
vcimgnetclient.py:330: PyGTKDeprecationWarning: Using positional arguments with the GObject constructor has been deprecated. Please specify keyword(s) for "type" or use a class specific constructor. See: https://wiki.gnome.org/PyGObject/InitializerDeprecations
  self.window =  Gtk.Window(Gtk.WindowType.TOPLEVEL)
vcimgnetclient.py:628: PyGTKDeprecationWarning: Using positional arguments with the GObject constructor has been deprecated. Please specify keyword(s) for "label" or use a class specific constructor. See: https://wiki.gnome.org/PyGObject/InitializerDeprecations
  mfQuit =  Gtk.MenuItem("Quit")
vcimgnetclient.py:629: PyGIDeprecationWarning: Using non GObject arguments for connect_object() is deprecated, use: connect_data(signal, callback, data, connect_flags=GObject.ConnectFlags.SWAPPED)
  mfQuit.connect_object("activate", lambda a: Gtk.main_quit(), "mfQuit")
vcimgnetclient.py:637: PyGTKDeprecationWarning: Using positional arguments with the GObject constructor has been deprecated. Please specify keyword(s) for "label" or use a class specific constructor. See: https://wiki.gnome.org/PyGObject/InitializerDeprecations
  bmf = Gtk.MenuItem("File")
vcimgnetclient.py:646: PyGTKDeprecationWarning: Using positional arguments with the GObject constructor has been deprecated. Please specify keyword(s) for "label" or use a class specific constructor. See: https://wiki.gnome.org/PyGObject/InitializerDeprecations
  mhAbout =  Gtk.MenuItem("About")
vcimgnetclient.py:654: PyGTKDeprecationWarning: Using positional arguments with the GObject constructor has been deprecated. Please specify keyword(s) for "label" or use a class specific constructor. See: https://wiki.gnome.org/PyGObject/InitializerDeprecations
  bmh = Gtk.MenuItem("Help")
Traceback (most recent call last):
  File "vcimgnetclient.py", line 812, in <module>
    Main()
  File "vcimgnetclient.py", line 719, in __init__
    Frontend.__init__(self, self.img_access)
  File "vcimgnetclient.py", line 301, in __init__
    self.winMain = self.WinMain(arg_img_access, self.dialog)
  File "vcimgnetclient.py", line 341, in __init__
    vbox.pack_start(self.menu_bar, False, False)
TypeError: Gtk.Box.pack_start() takes exactly 5 arguments (4 given)
4)

For ALL vbox.pack_start, add one parameter 0 at the end for each case. For instance:
vbox.pack_start(self.menu_bar, False, False)
to
vbox.pack_start(self.menu_bar, False, False, 0)

Oh my god … There are STILL SO MANY things to do, in order to make vcimgnetclient.py Python3 compatible. Therefore, I implement my own vcimgnetclient_qt.py based on PyQt5.

2.3.3 Longer Vision’s vcimgnetclient_qt.py

2.3.3.1 Server

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
22:42:22[root@VC-Z] /home/user
#whoami
root

22:42:24[root@VC-Z] /home/user
#vcimgnetsrv &
[1] 896

22:42:30[root@VC-Z] /home/user
#ps -e | grep vcimgnetsrv
  896 pts/0    00:00:00 vcimgnetsrv

22:42:40[root@VC-Z] /home/user
#vctp
Acquisiton&Copy Duration:          4s      39644us  for 100 Cycles ==  24.754656fps.

Acquisiton&Copy Duration:          4s      34709us  for 100 Cycles ==  24.784935fps.
......

2.3.3.2 Client

Sorry, I’m NOT going to show my code, but the performance can be demonstrated as follows:

Longer Vision Technology - Vision Component's Image Grabber - PyQt5

Posted on | Edited on | In ,

Oh-my-god, kaggle competition about diabetis… Hmmm, I have something to do these days… It’s also a GOOD chance for me to learn some medical terms… In this blog, we’re going to try out this kaggle competition - APTOS 2019 Blindness Detection.

1. Pre-processing Diabetic Retinopathy Images

It’s NOT hard for us to locate APTOS [UpdatedV14] Preprocessing- Ben’s & Cropping, which seems to me a professional snippet of code preprocessing diabetic retinopathy images. Let’s just try it out directly.

1.1 Implementation

The implementation is cited directly from APTOS [UpdatedV14] Preprocessing- Ben’s & Cropping, with trivial modifications. You are welcome to download my snippet of code here.

1.2 Show Off Intermediately Processed Images

5 Categories Diabetic Retinopathy Concepts
5 Categories Diabetic Retinopathy Concepts
Original Images in Color Original Images in Gray Scale
Original Images in Color Original Images in Gray Scale
Ben Graham’s Preprocessing for Gray Scale Images Ben Graham’s Preprocessing for Color Images
Ben Graham's Preprocessing for Gray Scale Images Ben Graham's Preprocessing for Color Images
Circle Crop Circle Crop High Resolution
Circle Crop Circle Crop High Resolution
Resized Train Resized Train High Resolution
Resized Train Resized Train High Resolution

Some extentions about diabetic retinopathy are given in the end:

Heatmap Inconsistency
Heatmap Inconsistency

Posted on | Edited on | In ,

Today’s weather is perfect for this party: annual summer alumni party of Wuhan University.

Alumni Me
Alumni Me

Today, I’ve got another GOOD news: our ARM Linux board based on AllWinner H3 is demonstrated successful.

1. Hardware

2.1 Appearance

Yeah, this is how OUR OWN board looks like:

Longer Vision Technology AllWinner H3 Board

2.2 Specifications

From the appearance, it’s NOT hard to tell the specifications:

Device Version Units
USB 2.0 2
Wired 100Mbps 1
HDMI 1
Power Adaptor 5V 1

2. Flash Armbian Debian

2.1 Armbian for AllWinner H3

Armbian provides abundant info for AllWinner H3:

2.2 Armbian for Orange Pi PC

I’m so lucky today that I tried ONLY ONE TIME and have this Debian Buster with Armbian Linux for Orange Pi PC successfully installed and run on our board. You just need to download and extract Armbian_5.91_Orangepipc_Debian_buster_next_4.19.59.img out, and have it installed by dd.

2.3 Longer Vision Technology H3 Board Test

It’s NOT hard to find out your board’s IP address

  • either from within your router
  • or boot your board with a monitor and run the command ip -c address.

Now, you should be able to SSH into your H3 baord directly as follows:

SSH Into LVT H3 Board

After you run

  • sudo apt update and sudo apt upgrade
  • sudo armbian-config

you should be able to upgrade your Debian Buster with Armbian Linux from 4.19.59-sunxi to 4.19.62-sunxi.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
longervision-GT72-6QE% ssh longervision@192.168.1.96
longervision@192.168.1.96's password: 
  ___  ____  _   ____   ____ 
 / _ \|  _ \(_) |  _ \ / ___|
| | | | |_) | | | |_) | |    
| |_| |  __/| | |  __/| |___ 
 \___/|_|   |_| |_|    \____|
                             
Welcome to Debian Buster with Armbian Linux 4.19.62-sunxi
System load:   0.16 0.12 0.08  	Up time:       6 min		
Memory usage:  18 % of 460MB  	IP:            192.168.1.96
CPU temp:      79°C           	
Usage of /:    2% of 58G    	

Last login: Sun Aug 25 20:13:29 2019 from 192.168.1.200

3. Issues

It’s a kind of late. Let me finish this up for now ASAP.

3.1 Hardware Issues

  • ONLY 1 USB 2.0 port is working properly.
  • It’s EASY to reach the critical temperature, as showing:

SSH Into LVT H3 Board

3.2

We really need a better low-cost chip to provide the support for:

  • USB 3.0
  • GigE

Posted on | Edited on | In ,

JeVois is an ARM Linux based camera module, delivered by JeVois Inc.. So far, this is the ONLY ARM Linux based camera module that I’m recommending, for its well-designed community as well as plenty of source code on Github.

1. Specifications

Clearly:

2. Getting Started with Jevois

The BEST place to start with JeVois is the following Youtube video:

I’m NOT going to repeat this step-by-step setup but move forward to the demonstration step directly.

3. Connect JeVois with Host

After having flashed JeVois’s TF card with the LATEST Linux image, we connect JeVois camera module into the laptop. This TF card will NOT mount onto the computer as a device automatically, but can be detected by lsusb as follows:

1
2
3
4
longervision-GT72-6QE% lsusb
......
Bus 001 Device 016: ID 1d6b:0102 Linux Foundation EEM Gadget
......

4. Simple Demonstration

4.1 Guvcview

Get Started Image Control Default Settings
JeVois Get Started JeVois Image Control
Video Control Settings 640x480 Video Control Settings 640x360
JeVois Video Control 640x480 JeVois Video Control 640x360

4.2 JeVois Inventor

4.2.1 JeVois Tutorial

Again, a Youtube video is suggested to follow for the demonstration:

4.2.2 My Test

You can either download JeVois Inventor directly from the Official Website, or like me, build JeVois Inventor source code from scratch, based on my own QT, etc. Anyway, after JeVois Inventor has been successfully installed, it’s the time for us to test it out.

JeVois Inventor Info JeVois Inventor Parameters
JeVois Inventor Info JeVois Inventor Parameters
JeVois Inventor Console JeVois Inventor Camera
JeVois Inventor Console JeVois Inventor Camera
JeVois Inventor Config JeVois Inventor Code C++
JeVois Inventor Config JeVois Inventor Code C++
JeVois Inventor Code Python DNN JeVois Inventor System
JeVois Inventor Code Python DNN JeVois Inventor System
JeVois Inventor System Export SD Card JeVois Inventor System Export SD Card Content
JeVois Inventor System Export SD Card JeVois Inventor System Export SD Card Content

Being quite busy these days. Seriously, ARM board based on AllWinner H3 from my company Longer Vision is manufactured, and under test. Let’s have some fun.

In the end, better to listen to some music while blogging.

Posted on | Edited on | In ,

0. Preface

My ESP32-Cam has finally arrived today. The brand of AI Thinker has been demonstrated to be reliable.

In fact, I strictly followed randomnerdtutorials’s blog ESP32-CAM Video Streaming and Face Recognition with Arduino IDE to demonstrate this low-cost video surveillance camera. So, in this blog, I ONLY enumerate some KEY POINTS that should be emphasized.

1. Connection

1.1 Schematic

ESP32-Cam can be connected to my laptop exactly the same as shown (cited directly from ESP32-CAM Video Streaming and Face Recognition with Arduino IDE).

ESP32-CAM-wiring-FTDI Schematic

1.2 Real Connection

The real connection looks as:

ESP32-CAM-wiring-FTDI Real Connection

1.3 lsusb

1
2
3
4
longervision-GT72-6QE% lsusb
......
Bus 001 Device 010: ID 0403:6001 Future Technology Devices International, Ltd FT232 Serial (UART) IC
......

2. Key Points on Flashing ESP32-Cam

2.1 Code Modification

Example code CameraWebServer should be trivially modified as:

1
2
3
4
5
6
// Select camera model
//#define CAMERA_MODEL_WROVER_KIT
//#define CAMERA_MODEL_ESP_EYE
//#define CAMERA_MODEL_M5STACK_PSRAM
//#define CAMERA_MODEL_M5STACK_WIDE
#define CAMERA_MODEL_AI_THINKER

2.2 Verify/Compile and Upload

2.2.1 Verify/Compile

1
2
Sketch uses 2241942 bytes (71%) of program storage space. Maximum is 3145728 bytes.
Global variables use 52696 bytes (16%) of dynamic memory, leaving 274984 bytes for local variables. Maximum is 327680 bytes.

2.2.2 Upload

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
esptool.py v2.6
Serial port /dev/ttyUSB0
Connecting....
Chip is ESP32D0WDQ5 (revision 1)
Features: WiFi, BT, Dual Core, 240MHz, VRef calibration in efuse, Coding Scheme None
MAC: c4:4f:33:13:32:7d
Uploading stub...
Running stub...
Stub running...
Configuring flash size...
Auto-detected Flash size: 4MB
Compressed 8192 bytes to 47...

Writing at 0x0000e000... (100 %)
Wrote 8192 bytes (47 compressed) at 0x0000e000 in 0.0 seconds (effective 4102.7 kbit/s)...
Hash of data verified.
Compressed 16832 bytes to 10888...

Writing at 0x00001000... (100 %)
Wrote 16832 bytes (10888 compressed) at 0x00001000 in 1.0 seconds (effective 138.0 kbit/s)...
Hash of data verified.
Compressed 2242064 bytes to 1795627...

Writing at 0x00010000... (0 %)
Writing at 0x00014000... (1 %)
Writing at 0x00018000... (2 %)
Writing at 0x0001c000... (3 %)
Writing at 0x00020000... (4 %)
Writing at 0x00024000... (5 %)
Writing at 0x00028000... (6 %)
Writing at 0x0002c000... (7 %)
Writing at 0x00030000... (8 %)
Writing at 0x00034000... (9 %)
Writing at 0x00038000... (10 %)
Writing at 0x0003c000... (10 %)
Writing at 0x00040000... (11 %)
Writing at 0x00044000... (12 %)
Writing at 0x00048000... (13 %)
Writing at 0x0004c000... (14 %)
Writing at 0x00050000... (15 %)
Writing at 0x00054000... (16 %)
Writing at 0x00058000... (17 %)
Writing at 0x0005c000... (18 %)
Writing at 0x00060000... (19 %)
Writing at 0x00064000... (20 %)
Writing at 0x00068000... (20 %)
Writing at 0x0006c000... (21 %)
Writing at 0x00070000... (22 %)
Writing at 0x00074000... (23 %)
Writing at 0x00078000... (24 %)
Writing at 0x0007c000... (25 %)
Writing at 0x00080000... (26 %)
Writing at 0x00084000... (27 %)
Writing at 0x00088000... (28 %)
Writing at 0x0008c000... (29 %)
Writing at 0x00090000... (30 %)
Writing at 0x00094000... (30 %)
Writing at 0x00098000... (31 %)
Writing at 0x0009c000... (32 %)
Writing at 0x000a0000... (33 %)
Writing at 0x000a4000... (34 %)
Writing at 0x000a8000... (35 %)
Writing at 0x000ac000... (36 %)
Writing at 0x000b0000... (37 %)
Writing at 0x000b4000... (38 %)
Writing at 0x000b8000... (39 %)
Writing at 0x000bc000... (40 %)
Writing at 0x000c0000... (40 %)
Writing at 0x000c4000... (41 %)
Writing at 0x000c8000... (42 %)
Writing at 0x000cc000... (43 %)
Writing at 0x000d0000... (44 %)
Writing at 0x000d4000... (45 %)
Writing at 0x000d8000... (46 %)
Writing at 0x000dc000... (47 %)
Writing at 0x000e0000... (48 %)
Writing at 0x000e4000... (49 %)
Writing at 0x000e8000... (50 %)
Writing at 0x000ec000... (50 %)
Writing at 0x000f0000... (51 %)
Writing at 0x000f4000... (52 %)
Writing at 0x000f8000... (53 %)
Writing at 0x000fc000... (54 %)
Writing at 0x00100000... (55 %)
Writing at 0x00104000... (56 %)
Writing at 0x00108000... (57 %)
Writing at 0x0010c000... (58 %)
Writing at 0x00110000... (59 %)
Writing at 0x00114000... (60 %)
Writing at 0x00118000... (60 %)
Writing at 0x0011c000... (61 %)
Writing at 0x00120000... (62 %)
Writing at 0x00124000... (63 %)
Writing at 0x00128000... (64 %)
Writing at 0x0012c000... (65 %)
Writing at 0x00130000... (66 %)
Writing at 0x00134000... (67 %)
Writing at 0x00138000... (68 %)
Writing at 0x0013c000... (69 %)
Writing at 0x00140000... (70 %)
Writing at 0x00144000... (70 %)
Writing at 0x00148000... (71 %)
Writing at 0x0014c000... (72 %)
Writing at 0x00150000... (73 %)
Writing at 0x00154000... (74 %)
Writing at 0x00158000... (75 %)
Writing at 0x0015c000... (76 %)
Writing at 0x00160000... (77 %)
Writing at 0x00164000... (78 %)
Writing at 0x00168000... (79 %)
Writing at 0x0016c000... (80 %)
Writing at 0x00170000... (80 %)
Writing at 0x00174000... (81 %)
Writing at 0x00178000... (82 %)
Writing at 0x0017c000... (83 %)
Writing at 0x00180000... (84 %)
Writing at 0x00184000... (85 %)
Writing at 0x00188000... (86 %)
Writing at 0x0018c000... (87 %)
Writing at 0x00190000... (88 %)
Writing at 0x00194000... (89 %)
Writing at 0x00198000... (90 %)
Writing at 0x0019c000... (90 %)
Writing at 0x001a0000... (91 %)
Writing at 0x001a4000... (92 %)
Writing at 0x001a8000... (93 %)
Writing at 0x001ac000... (94 %)
Writing at 0x001b0000... (95 %)
Writing at 0x001b4000... (96 %)
Writing at 0x001b8000... (97 %)
Writing at 0x001bc000... (98 %)
Writing at 0x001c0000... (99 %)
Writing at 0x001c4000... (100 %)
Wrote 2242064 bytes (1795627 compressed) at 0x00010000 in 159.6 seconds (effective 112.4 kbit/s)...
Hash of data verified.
Compressed 3072 bytes to 134...

Writing at 0x00008000... (100 %)
Wrote 3072 bytes (134 compressed) at 0x00008000 in 0.0 seconds (effective 1534.2 kbit/s)...
Hash of data verified.

Leaving...
Hard resetting via RTS pin...

2.3 Disconnect GPIO 0 from GND

Before testing the code, be sure to disconnect GPIO 0 from GND.

ESP32-CAM Disconnect

2.4 RESET ESP32-Cam Board

It’s VERY important that you open the Serial Monitor dialog to monitor the serial port. You will see the following outputs after resetting ESP32-Cam board.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
ets Jun  8 2016 00:22:57

rst:0x1 (POWERON_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:1
load:0x3fff0018,len:4
load:0x3fff001c,len:1100
load:0x40078000,len:9232
load:0x40080400,len:6400
entry 0x400806a8

[E][sccb.c:154] SCCB_Write(): SCCB_Write Failed addr:0x30, reg:0xff, data:0x01, ret:-1
.........
WiFi connected
Starting web server on port: '80'
Starting stream server on port: '81'
Camera Ready! Use 'http://192.168.1.94' to connect

2.5 Video Streaming

By visiting the allocated IP address within any web browser, you’ll see the following picture:

ESP32-CAM OV2460

And you’ll notice the streaming info from serial port:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
ets Jun  8 2016 00:22:57

rst:0x1 (POWERON_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:1
load:0x3fff0018,len:4
load:0x3fff001c,len:1100
load:0x40078000,len:9232
load:0x40080400,len:6400
entry 0x400806a8

[E][sccb.c:154] SCCB_Write(): SCCB_Write Failed addr:0x30, reg:0xff, data:0x01, ret:-1
.........
WiFi connected
Starting web server on port: '80'
Starting stream server on port: '81'
Camera Ready! Use 'http://192.168.1.94' to connect
MJPG: 5297B 63ms (15.9fps), AVG: 63ms (15.9fps), 0+0+0+0=0 0
MJPG: 5301B 40ms (25.0fps), AVG: 51ms (19.6fps), 0+0+0+0=0 0
MJPG: 5460B 39ms (25.6fps), AVG: 47ms (21.3fps), 0+0+0+0=0 0
MJPG: 5301B 40ms (25.0fps), AVG: 45ms (22.2fps), 0+0+0+0=0 0
MJPG: 5460B 43ms (23.3fps), AVG: 45ms (22.2fps), 0+0+0+0=0 0
MJPG: 5374B 41ms (24.4fps), AVG: 44ms (22.7fps), 0+0+0+0=0 0
MJPG: 5460B 37ms (27.0fps), AVG: 43ms (23.3fps), 0+0+0+0=0 0
MJPG: 5464B 43ms (23.3fps), AVG: 43ms (23.3fps), 0+0+0+0=0 0
MJPG: 5460B 37ms (27.0fps), AVG: 42ms (23.8fps), 0+0+0+0=0 0
MJPG: 5464B 46ms (21.7fps), AVG: 42ms (23.8fps), 0+0+0+0=0 0
MJPG: 5460B 30ms (33.3fps), AVG: 41ms (24.4fps), 0+0+0+0=0 0
MJPG: 5464B 43ms (23.3fps), AVG: 41ms (24.4fps), 0+0+0+0=0 0
MJPG: 5460B 39ms (25.6fps), AVG: 41ms (24.4fps), 0+0+0+0=0 0
MJPG: 5464B 37ms (27.0fps), AVG: 41ms (24.4fps), 0+0+0+0=0 0
MJPG: 5460B 37ms (27.0fps), AVG: 41ms (24.4fps), 0+0+0+0=0 0
......

3. Multiple Camera Streaming with Static IPs

I’m NOT going to talk about how to set up static IPs for IP cameras (well, yes, clearly, ESP32-Cam can be looked on as a low-cost video surveillance IP camera for sure). We ONLY demonstrate 4 ESP32-Cams are under streaming simultaneously with 4 statically allocated IP addresses.

We show the capturing effects FIRST:

4 ESP32-Cam Capturing Simultaneously

Then, we show how the cables are connected by using a power bank and an extra bread board. Here, FTDI is just used as a power adaptor to provide 5V voltage.

4 ESP32-Cams Connected to Power Bank

4 ESP32-Cams Cable Connection

Posted on | Edited on | In 

        

        

噩耗传来，今晨，高中同学逝世，年仅43岁。今天是2019年8月15日，是日军投降的日子。同学经历4个月急性胰腺炎的煎熬，耗光所有家财，于北京时间早上9时11分不治。这两个神奇的时间…


难忘高中一起踢球、打麻将的日子。在球场上，他是跑不死的。而且身体肌肉比较多，所以才有两个外号：“施瓦辛格”、“周扎实”。


我的印象，他应该是我们高中班第一位走的同学。人生无常，想当年，他也是我们班第一位获得保送的同学吧？老天爷就是这样不公平。1995年，湖北省是高考前先保送，但是保送了之后还要参加高考，保送者确定必须过一本线才能进入所保送的大学就读。他心态好的出奇，保送东北大学之后就不再学习了，每天看金庸小说。高考竟然考到652？不太记得分数。总之，是一个在湖北省稳稳上清华的分数。经历了本硕连读，东大阿派的锤炼，他成功完成香港中文大学博士学位，并令人羡慕的进入珠海某大学就职，面朝大海、春暖花开的海景房，曾经令我幻想羡慕不已…可惜至今，我也没有机会去看过。


他的婚礼是在武汉办的，我是亲自参加的，他娶了一位非常漂亮的嫂子。他在婚礼上非常紧张，大概是兴奋的？夫妻对拜的时候，竟然说出“感谢娶我为妻”！这个证据确凿，每每想起，忍俊不止。


我在英国的时候，他给我介绍了一个很漂亮的女朋友。可惜缘分没有让我们走到一起。我还是通过这位已经是两个孩子的妈妈，获悉同学的病情，并立即通知高中同班同学筹款。只可惜还是…听到病危病重，真的是不敢相信自己的耳朵！确定水滴筹的照片就是他本人，已经瘦得不成人形！获悉同学病情的前三天，每天早上起床，眼睛湿润都不自知…前几年我到珠海，我们几位高中同学还吃海鲜聚会，没有见到他。没想到那次错过，就错过了一世…


这几年，见得太多了！这个社会，好像真的是：越是好人，越走得早？老天爷啊…“逢庙必拜、遇观必香”…人到中年，真的开始有点相信神佛…根本还没来得及小小欢庆几小时前首次带父母到美国小游，就…明天去列治文拜佛祈福。


我很喜欢卡拉OK，一直想要在hexo blog 加入自己原唱作为背景音乐（这也算是创新吧？），今天这篇不算是祷文的blog，纪念我同学，也算督促我完成了这个小小任务。回到家、刚刚开始check wechat，就看到这个噩耗，真的是… 这是我第一次发全民K歌，中间有跑调，我觉得没有必要修改任何！因为这就是我自己原唱，在听到消息之后，也着实情绪不能控制…


人生的意义在什么？“且行且珍惜”！珍惜身边拥有的一切吧！想起去年某台姐对我说：“看到你一家三口团聚，就感觉很幸福”。是啊…台姐本人从小姥姥带大、10岁出头一个人给姥姥送葬；而周伯伯白发人送黑发人，这…人，真的不能太“不知足”！我思想还没成熟呢！这么大了，怎么长的？太幼稚！！种地，有什么不好？？


不知不觉，温哥华当地时间2019年8月15日凌晨3点20分。愿你在天堂没有痛苦…